#weight-sparse transformers15/11/2025
OpenAI Trains Weight-Sparse Transformers to Reveal Compact, Interpretable Circuits
'OpenAI enforces extreme weight sparsity in GPT-2 style transformers to recover compact, verifiable circuits that explain specific model behaviors on algorithmic Python tasks.'